Advances In Natural Language Processing Research Articles

Enhancing smart contract security: Leveraging pre‐trained language models for advanced vulnerability detection

AbstractThe burgeoning interest in decentralized applications (Dapps), spurred by advancements in blockchain technology, underscores the critical role of smart contracts. However, many Dapp users, often without deep knowledge of smart contracts, face financial risks due to hidden vulnerabilities. Traditional methods for detecting these vulnerabilities, including manual inspections and automated static analysis, are plagued by issues such as high rates of false positives and overlooked security flaws. To combat this, the article introduces an innovative approach using the bidirectional encoder representations from transformers (BERT)‐ATT‐BiLSTM model for identifying potential weaknesses in smart contracts. This method leverages the BERT pre‐trained model to discern semantic features from contract opcodes, which are then refined using a Bidirectional Long Short‐Term Memory Network (BiLSTM) and augmented by an attention mechanism that prioritizes critical features. The goal is to improve the model's generalization ability and enhance detection accuracy. Experiments on various publicly available smart contract datasets confirm the model's superior performance, outperforming previous methods in key metrics like accuracy, F1‐score, and recall. This research not only offers a powerful tool to bolster smart contract security, mitigating financial risks for average users, but also serves as a valuable reference for advancements in natural language processing and deep learning.

Knowledge sharing in manufacturing using LLM-powered tools: user study and model benchmarking.

Recent advances in natural language processing enable more intelligent ways to support knowledge sharing in factories. In manufacturing, operating production lines has become increasingly knowledge-intensive, putting strain on a factory's capacity to train and support new operators. This paper introduces a Large Language Model (LLM)-based system designed to retrieve information from the extensive knowledge contained in factory documentation and knowledge shared by expert operators. The system aims to efficiently answer queries from operators and facilitate the sharing of new knowledge. We conducted a user study at a factory to assess its potential impact and adoption, eliciting several perceived benefits, namely, enabling quicker information retrieval and more efficient resolution of issues. However, the study also highlighted a preference for learning from a human expert when such an option is available. Furthermore, we benchmarked several commercial and open-sourced LLMs for this system. The current state-of-the-art model, GPT-4, consistently outperformed its counterparts, with open-source models trailing closely, presenting an attractive option given their data privacy and customization benefits. In summary, this work offers preliminary insights and a system design for factories considering using LLM tools for knowledge management.

Self-supervised text de-stylization based on BERT

Recent advancements in Natural Language Processing (NLP) have ushered in a new era of textual style transfer (TST), a domain aimed at altering textual attributes such as tone and sentiment while preserving the content's essence. This study introduces a creative framework that employs a dual-component architecture consisting of a classifier and a generator to achieve text de-stylization, particularly sentiment neutralization. The classifier, built upon the Bidirectional Encoder Representations from Transformers (BERT) model, serves as a dynamic loss function guiding the generator, constructed on a Transformer-based encoder-decoder framework, to produce sentiment-neutral text. Our method leverages a self-supervised mechanism, enabling the generation of target text without reliance on parallel corpora, thereby addressing the limitations of existing TST methodologies. We preprocessed datasets from Stanford Sentiment Treebank-5 (SST-5) and Internet Movie Database (IMDb) movie reviews and employed them for training the classifier and generator, respectively. Preliminary results demonstrate the model's proficiency in preserving semantic integrity while effectively neutralizing sentiment. Future work envisions expanding this framework to enable text stylization across a spectrum of discursive contexts, enhanced by deep learning architectures and an iterative feedback mechanism for user-driven refinement.

The Generalization and Robustness of Transformer-Based Language Models on Commonsense Reasoning

The advent of powerful transformer-based discriminative language models and, more recently, generative GPT-family models, has led to notable advancements in natural language processing (NLP), particularly in commonsense reasoning tasks. One such task is commonsense reasoning, where performance is usually evaluated through multiple-choice question-answering benchmarks. Till date, many such benchmarks have been proposed and `leaderboards' tracking state-of-the-art performance on those benchmarks suggest that transformer-based models are approaching human-like performance. However, due to documented problems such as hallucination and bias, the research focus is shifting from merely quantifying accuracy on the task to an in-depth, context-sensitive probing of LLMs' generalization and robustness. To gain deeper insight into diagnosing these models' performance in commonsense reasoning scenarios, this thesis addresses three main studies: the generalization ability of transformer-based language models on commonsense reasoning, the trend in confidence distribution of these language models confronted with ambiguous inference tasks, and a proposed risk-centric evaluation framework for both discriminative and generative language models.

Test-Time Personalization with Meta Prompt for Gaze Estimation

Despite the recent remarkable achievement in gaze estimation, efficient and accurate personalization of gaze estimation without labels is a practical problem but rarely touched on in the literature. To achieve efficient personalization, we take inspiration from the recent advances in Natural Language Processing (NLP) by updating a negligible number of parameters, "prompts", at the test time. Specifically, the prompt is additionally attached without perturbing original network and can contain less than 1% of a ResNet-18's parameters. Our experiments show high efficiency of the prompt tuning approach. The proposed one can be 10 times faster in terms of adaptation speed than the methods compared. However, it is non-trivial to update the prompt for personalized gaze estimation without labels. At the test time, it is essential to ensure that the minimizing of particular unsupervised loss leads to the goals of minimizing gaze estimation error. To address this difficulty, we propose to meta-learn the prompt to ensure that its updates align with the goal. Our experiments show that the meta-learned prompt can be effectively adapted even with a simple symmetry loss. In addition, we experiment on four cross-dataset validations to show the remarkable advantages of the proposed method.

Abstract 3528: Constructing knowledge graph of N6-methyladnosion regulations in cancer from literature using ChatGPT-4

Abstract N6-methyladenosine (m6A) is the most abundant mRNA modification with a crucial role in cellular processes. Its involvement in cancers is well-established. Dysregulation of m6A is linked to cancer initiation, progression, and therapy resistance, impacting the tumor microenvironment (TME) and promoting metastasis. Understanding m6A's regulatory role in cancer is essential for identifying new therapeutic targets. We considered the construction of a knowledge graph (KG) representing molecular regulatory mechanisms (MRMs) of m6A in healthy conditions and different cancers from PubMed papers. Such a KG consists of interconnected triplets of (head node, regulatory relationship, tail node), each denoting regulation between genes or a gene and a phenotype/process. Advances in natural language processing (NLP) have provided tools such as SemRep and GNBR to automate the extraction of these triplets from the literature. However, these methods don’t extract contextual information, such as the type of cancers or cells in which these MRMs occur, leading to contradictory regulations in the constructed KG. In addition, existing approaches were designed to extract generic regulatory relations and therefore struggled in properly capturing unique concepts associated with m6A regulatory mechanisms. To tackle this challenge, we designed a novel ChatGPT-4 prompt to extract relational graphs from papers tailored for m6A regulatory mechanisms. An annotated dataset of 400 titles on m6A-related MRMs were created to evaluate the performance. We then applied the prompt to 1023 papers to create a m6A Molecular Regulatory Mechanism Knowledge Graph (m6A-MRM-KG). This graph illuminates m6A's roles in gene expression regulation, especially in cancer and immunity. Anticipated to enhance our understanding of cancer development, it provides insights into potential therapeutic strategies. The m6A-MRM-KG not only organizes information but also empowers researchers to uncover novel insights in the dynamic relationships of m6A modifications in diverse biological contexts. Citation Format: Sumin Jo, Xidong Wu, Yiming Zeng, Arun Das, Ting-He Zhang, David Alexander Spellman, Adam Ferris, Shou-Jiang Gao, Jianqiu (Michelle) Zhang, Yu-Chiao Chiu, Yufei Huang. Constructing knowledge graph of N6-methyladnosion regulations in cancer from literature using ChatGPT-4 [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3528.

From McCulloch to GPT - 4: stages of development of artificial intelligence.

The article examines the history of the development of artificial intelligence (AI), starting from its first theoretical and practical steps and tracing the evolution to modern achievements. The article provides an overview of the key milestones, scientific discoveries and technological breakthroughs made in the field of AI. The most important figures, ideas and principles that influenced its development are also discussed. In the context of this development, various definitions of artificial intelligence are given. There are several key stages in the history of AI: the early stages, the quiet period, the AI renaissance, and the era of AI in the new millennium. Each of these stages made its own unique contribution to the progress of AI. The modern period is characterized by rapid development, especially in the field of machine learning and deep learning. These methods allow artificial intelligence to learn from data and identify complex patterns. Advances in natural language processing, such as models GPT and its modifications, have shown outstanding results. However, despite linguistic advances, GPT remains limited in aspects important to creating strong AI. The article discusses the limitations of modern language models, as well as the prerequisites and prospects for the development of strong artificial intelligence. Special attention is paid to the project of Elon Musk, who, having launched the company X.AI, is engaged in research in the field of creating strong AI with the goal of “knowledge of reality.” The article also proposes an alternative approach to creating strong artificial intelligence - the development of an artificial brain based on a multidimensional multi-connected receptor-effector neuron-like growing network. Some aspects of the emergence of artificial consciousness are also considered.

Natural language instructions induce compositional generalization in networks of neurons.

A fundamental human cognitive feat is to interpret linguistic instructions in order to perform novel tasks without explicit task experience. Yet, the neural computations that might be used to accomplish this remain poorly understood. We use advances in natural language processing to create a neural model of generalization based on linguistic instructions. Models are trained on a set of common psychophysical tasks, and receive instructions embedded by a pretrained language model. Our best models can perform a previously unseen task with an average performance of 83% correct based solely on linguistic instructions (that is, zero-shot learning). We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic representations of instructions, allowing language to cue the proper composition of practiced skills in unseen settings. We show how this model generates a linguistic description of a novel task it has identified using only motor feedback, which can subsequently guide a partner model to perform the task. Our models offer several experimentally testable predictions outlining how linguistic information must be represented to facilitate flexible and general cognition in the human brain.

Disease progression strikingly differs in research and real-world Parkinson’s populations

Characterization of Parkinson’s disease (PD) progression using real-world evidence could guide clinical trial design and identify subpopulations. Efforts to curate research populations, the increasing availability of real-world data, and advances in natural language processing, particularly large language models, allow for a more granular comparison of populations than previously possible. This study includes two research populations and two real-world data-derived (RWD) populations. The research populations are the Harvard Biomarkers Study (HBS, N = 935), a longitudinal biomarkers cohort study with in-person structured study visits; and Fox Insights (N = 36,660), an online self-survey-based research study of the Michael J. Fox Foundation. Real-world cohorts are the Optum Integrated Claims-electronic health records (N = 157,475), representing wide-scale linked medical and claims data and de-identified data from Mass General Brigham (MGB, N = 22,949), an academic hospital system. Structured, de-identified electronic health records data at MGB are supplemented using a manually validated natural language processing with a large language model to extract measurements of PD progression. Motor and cognitive progression scores change more rapidly in MGB than HBS (median survival until H&Y 3: 5.6 years vs. >10, p < 0.001; mini-mental state exam median decline 0.28 vs. 0.11, p < 0.001; and clinically recognized cognitive decline, p = 0.001). In real-world populations, patients are diagnosed more than eleven years later (RWD mean of 72.2 vs. research mean of 60.4, p < 0.001). After diagnosis, in real-world cohorts, treatment with PD medications has initiated an average of 2.3 years later (95% CI: [2.1–2.4]; p < 0.001). This study provides a detailed characterization of Parkinson’s progression in diverse populations. It delineates systemic divergences in the patient populations enrolled in research settings vs. patients in the real-world. These divergences are likely due to a combination of selection bias and real population differences, but exact attribution of the causes is challenging. This study emphasizes a need to utilize multiple data sources and to diligently consider potential biases when planning, choosing data sources, and performing downstream tasks and analyses.

The End of the Policy Analyst? Testing the Capability of Artificial Intelligence to Generate Plausible, Persuasive, and Useful Policy Analysis

Policy advising in government centers on the analysis of public problems and the developing of recommendations for dealing with them. In carrying out this work, policy analysts consult a variety of sources and work to synthesize that body of evidence into useful decision support documents commonly called briefing notes. Advances in natural language processing (NLP) have led to the continuing development of tools that can undertake a similar task. Given a brief prompt, a large language model (LLM) can synthesize information in content databases. This article documents the findings from an experiment that tested whether contemporary NLP technology is capable of producing public policy relevant briefing notes that expert evaluators judge to be useful. The research involved two stages. First, briefing notes were created using three models: NLP generated; human generated; and NLP generated/human edited. Next, two panels of retired senior public servants (with only one panel informed of the use of NLP in the experiment) were asked to judge the briefing notes using a heuristic evaluation rubric. The findings indicate that contemporary NLP tools were not able to, on their own, generate useful policy briefings. However, the feedback from the expert evaluators indicates that automatically generated briefing notes might serve as a useful supplement to the work of human policy analysts. And the speed with which the capabilities of NLP tools are developing, supplemented with access to a larger corpus of previously prepared policy briefings and other policy-relevant material, suggests that the quality of automatically generated briefings may improve significantly in the coming years. The article concludes with reflections on what such improvements might mean for the future practice of policy analysis.

Leveraging natural language processing to study emotional coherence in psychotherapy.

The association between emotional experience and expression, known as emotional coherence, is considered important for individual functioning. Recent advances in natural language processing (NLP) make it possible to automatically recognize verbally expressed emotions in psychotherapy dialogues and to explore emotional coherence with larger samples and finer granularity than previously. The present study used state-of-the-art emotion recognition models to automatically label clients' emotions at the utterance level, employed these labeled data to examine the coherence between verbally expressed emotions and self-reported emotions, and examined the associations between emotional coherence and clients' improvement in functioning throughout treatment. The data comprised 872 transcribed sessions from 68 clients. Clients self-reported their functioning before each session and their emotions after each. A subsample of 196 sessions were manually coded. A transformer-based approach was used to automatically label the remaining data for a total of 139,061 utterances. Multilevel modeling was used to assess emotional coherence and determine whether it was associated with changes in clients' functioning throughout treatment. The emotion recognition model demonstrated moderate performance. The findings indicated a significant association between verbally expressed emotions and self-reported emotions. Coherence in clients' negative emotions was associated with improvement in functioning. The results suggest an association between clients' subjective experience and their verbal expression of emotions and underscore the importance of this coherence to functioning. NLP may uncover crucial emotional processes in psychotherapy. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The general intelligence of GPT–4, its knowledge diffusive and societal influences, and its governance

Recent breakthroughs in artificial intelligence (AI) research include advancements in natural language processing (NLP) achieved by large language models (LLMs), and; in particu- lar, generative pre–trained transformer (GPT) architectures. The latest GPT developed by OpenAI, GPT–4, has shown re- markable intelligence capabilities across various domains and tasks. It exhibits capabilities in abstraction, comprehension, vision, computer coding, mathematics, and more, suggesting it to be a significant step towards artificial general intelligence (AGI); a level of AI that possesses capabilities similar to hu- man intelligence. In this paper we (1) review the capabili- ties GPT–4 demonstrates in the above–mentioned areas, (2) study some drawbacks of autoregressive architectures, (3) high- light some areas where GPT–4 can be improved, along also with some fundamental questions on how and why this LLM achieves the intelligence it has, (4) present a potential path which could probably lead the advancement of GPT models towards super–human domain intelligence, realized by the de- velopment of products such as auto–researchers, (5) show how GPT–4 can facilitate the diffusion of knowledge across differ- ent areas of science by promoting their interpretability and explainability (IE) to inexperts, (6) discuss a broad range of influences deployment of this powerful technology will have on the society, (7) address the governance of human–competitive AI, and; finally, (8) hint on the benchmarking methods used to evaluate the capabilities of LLMs, as well as on some challenges in defining and measuring these capabilities in the framework of AGI. Where applicable, the topics are accompanied by their specific potential implications on medical imaging.

Artificial Intelligence and IBD: Where are We Now and Where Will We Be in the Future?

Artificial intelligence (AI) is quickly demonstrating the ability to address problems and challenges in the care of IBD. This review with commentary will highlight today's advancements in AI applications for IBD in image analysis, understanding text, and replicating clinical knowledge and experience. Advancements in machine learning methods, availability of high-performance computing, and increasing digitization of medical data are providing opportunities for AI to assist in IBD care. Multiple groups have demonstrated the ability of AI to replicate expert endoscopic scoring in IBD, with expansion into automated capsule endoscopy, enterography, and histologic interpretations. Further, AI image analysis is being used to develop new endoscopic scoring with more granularity and detail than is possible using conventional methods. Advancements in natural language processing are proving to reduce laborious tasks required in the care of IBD, including documentation, information searches, and chart review. Finally, large language models and chatbots that can understand language and generate human-like replies are beginning to exhibit clinical intelligence that will revolutionize how we deliver IBD care. Today, AI is being deployed to replicate expert judgement in specific tasks where disagreement, subjectivity, and bias are common. However, the near future will herald contributions of AI doing what we cannot, including new detailed measures of IBD, enhanced analysis of images, and perhaps even fully automating care. As we speculate on future technologic capabilities that may improve how we care for IBD, this review will also consider how we will implement and fairly use AI in practice.

GPT classifications, with application to credit lending

Generative Pre-trained Transformers (GPT) and Large language models (LLMs) have made significant advancements in natural language processing in recent years. The practical applications of LLMs are undeniable, rendering moot any debate about their impending influence. The power of LLMs has made them similar to machine learning models for decision-making problems. In this paper, we focus on binary classification which is a common use of ML models, particularly in credit lending applications. We show how a GPT model can perform almost as accurately as a classical logistic machine learning model but with a much lower number of sample observations. In particular, we show how, in the context of credit lending, LLMs can be improved and reach performances similar to classical logistic regression models using only a small set of examples.

A Survey on Challenges and Advances in Natural Language Processing with a Focus on Legal Informatics and Low-Resource Languages

The field of Natural Language Processing (NLP) has experienced significant growth in recent years, largely due to advancements in Deep Learning technology and especially Large Language Models. These improvements have allowed for the development of new models and architectures that have been successfully applied in various real-world applications. Despite this progress, the field of Legal Informatics has been slow to adopt these techniques. In this study, we conducted an extensive literature review of NLP research focused on legislative documents. We present the current state-of-the-art NLP tasks related to Law Consolidation, highlighting the challenges that arise in low-resource languages. Our goal is to outline the difficulties faced by this field and the methods that have been developed to overcome them. Finally, we provide examples of NLP implementations in the legal domain and discuss potential future directions.

Natural Language Processing for Radiation Oncology: Personalizing Treatment Pathways.

Natural language processing (NLP), a technology that translates human language into machine-readable data, is revolutionizing numerous sectors, including cancer care. This review outlines the evolution of NLP and its potential for crafting personalized treatment pathways for cancer patients. Leveraging NLP's ability to transform unstructured medical data into structured learnable formats, researchers can tap into the potential of big data for clinical and research applications. Significant advancements in NLP have spurred interest in developing tools that automate information extraction from clinical text, potentially transforming medical research and clinical practices in radiation oncology. Applications discussed include symptom and toxicity monitoring, identification of social determinants of health, improving patient-physician communication, patient education, and predictive modeling. However, several challenges impede the full realization of NLP's benefits, such as privacy and security concerns, biases in NLP models, and the interpretability and generalizability of these models. Overcoming these challenges necessitates a collaborative effort between computer scientists and the radiation oncology community. This paper serves as a comprehensive guide to understanding the intricacies of NLP algorithms, their performance assessment, past research contributions, and the future of NLP in radiation oncology research and clinics.

Words of a Leader: The Importance of Intersectionality for Understanding Women Leaders’ Use of Dominant Language and How Others Receive It

Management scholars have long examined gender disparities in leaders’ communication and followers’ reactions. There is, however, a paucity of research that takes an intersectional perspective. This article takes that step, using an intersectional lens to examine women leaders’ use of dominant language and how others receive it. Leveraging advances in natural-language processing, I analyzed the stereotype content of more than 250,000 Congressional remarks (Study 1) and almost one million tweets (Study 2) by leaders. Women leaders referenced dominance more than men did (using more words like “powerful”), violating stereotypes that depict women as submissive. However, as theory on racialized gender stereotypes suggests, this effect was unique to White leaders. Two additional studies revealed backlash to women leaders’ use of dominant language. Analyzing almost 18,000 editorials revealed the more that women leaders referenced dominance, the more they were portrayed as dominant but also cold. Effects were strongest for Black and Latina women (Study 3). Finally, an experiment using simulated social media profiles found the more that Black women (but not men) leaders referenced dominance, the more voters rated them as less likeable, a result that was unique to Black leaders (Study 4). The article demonstrates the critical importance of intersectionality for understanding gender inequality in leaders’ communication and its reception by the media and the public.

AI in Autonomous Vehicles

The integration of Artificial Intelligence (AI) in autonomous vehicles represents a transformative leap towards safer, more efficient, and technologically advanced transportation systems. This abstract provides a comprehensive overview of the dynamic landscape where AI converges with autonomous vehicles, examining the synergies, challenges, and far-reaching implications of this groundbreaking integration AI-Powered Perception and Decision-Making: Delving into the technological core, the abstract explores how AI empowers autonomous vehicles with sophisticated perception systems, such as computer vision and sensor fusion. It discusses the role of machine learning algorithms in real-time decision-making, enabling vehicles to adapt to dynamic road conditions and unforeseen circumstances. Challenges in Autonomy: Recognizing the complexity of autonomous systems, the abstract addresses challenges such as handling edge cases, ensuring robustness against adversarial attacks, and navigating regulatory and ethical considerations. It emphasizes the importance of addressing these challenges to foster public trust and acceptance of autonomous vehicles. Human-AI Interaction in Autonomous Vehicles: Examining the interface between humans and AI-driven vehicles, the abstract discusses the importance of designing intuitive and trustworthy communication channels. It explores advancements in natural language processing and gesture recognition, fostering seamless collaboration between humans and autonomous systems. Regulatory Landscape and Ethical Considerations: Recognizing the pivotal role of regulations, the abstract discusses the evolving regulatory landscape for autonomous vehicles. It delves into ethical considerations surrounding AI decisions in critical situations, underscoring the need for a harmonized approach to ensure responsible AI deployment in autonomous driving scenarios.

Exploring AI-mediated informal digital learning of English (AI-IDLE): a mixed-method investigation of Chinese EFL learners’ AI adoption and experiences

Recent advancements in natural language processing and large language models have ushered language learning into the age of artificial intelligence (AI). Recognizing the affordances of generative AI tools, this paper aims to examine the degree to which L2 learners accepted and leveraged large language model platforms (e.g. ChatGPT, Bing Chat) for the informal digital learning of English (IDLE) purposes. Employing an explanatory sequential mixed-method design, this study draws on the technology acceptance model (TAM) and collects data via an adapted TAM questionnaire and an interview guide. A total of 867 Chinese EFL (English as a foreign language) learners answered the online survey, while 20 attended the post-survey interviews. Drawing on a validated structural model that elucidates the inter-factor relationships of perceived ease of use, perceived usefulness, intention to use, and actual use, the quantitative analysis provides statistical accounts for EFL learners’ adoption of Generative Pre-trained Transformer (GPT) technologies. The qualitative findings, derived from the interview data, reveal three key themes: (1) how perceived usefulness of chatbots for IDLE emerges from hands-on experimentation with these tools; (2) how intention to use increases as learners negotiate chatbot affordances and constraints; and (3) how actual use of chatbots for IDLE involves using these tools as tutors or conversation partners. Connections between quantitative and qualitative findings enhance our understanding of how EFL learners negotiate the affordances and constraints of highly capable AI technologies to participate in creative IDLE practices. By understanding these practices, this study draws attention to the attitudes and practices that constitute AI literacies, ultimately offering implications for future classroom practices and research.

NodeVector: A Novel Network Node Vectorization with Graph Analysis and Deep Learning

Network node embedding captures structural and relational information of nodes in the network and allows for us to use machine learning algorithms for various prediction tasks on network data that have an inherently complex and disordered structure. Network node embedding should preserve as much information as possible about important network properties where information is stored, such as network structure and node properties, while representing nodes as numerical vectors in a lower-dimensional space than the original higher dimensional space. Superior node embedding algorithms are a powerful tool for machine learning with effective and efficient node representation. Recent research in representation learning has led to significant advances in automating features through unsupervised learning, inspired by advances in natural language processing. Here, we seek to improve the representation quality of node embeddings with a new node vectorization technique that uses network analysis to overcome network-based information loss. In this study, we introduce the NodeVector algorithm, which combines network analysis and neural networks to transfer information from the target network to node embedding. As a proof of concept, our experiments performed on different categories of network datasets showed that our method achieves better results than its competitors for target networks. This is the first study to produce node representation by unsupervised learning using the combination of network analysis and neural networks to consider network data structure. Based on experimental results, the use of network analysis, complex initial node representation, balanced negative sampling, and neural networks has a positive effect on the representation quality of network node embedding.